In network based environments, load balancers are used to distribute workload across multiple computers. This is often done in order to achieve optimal resource utilization, maximize throughput and avoid overload. In the context of a cluster network that serves client requests, a load balancer typically receives a request from a client and relays it to a particular backend server in the cluster responsible for processing that request and issuing a response. Load balancers can be implemented as hardware, software or some combination thereof and usually attempt to distribute the request load evenly across all of the servers in an effort to prevent any individual server in the cluster from being overloaded with requests.
There exist a number of host selection algorithms which load balancers use to determine which backend server to send the request to. Some of the simple algorithms include round-robin and random selection of hosts, while more sophisticated algorithms may consider other factors, such as how busy a server may be at the time of distributing the request and the like. For example, the least-connections algorithm may distribute the incoming request to the server having the fewest number of open connections or at least assign a higher preference to servers having fewer numbers of connections.
While load balancers generally serve their purpose well, certain situations may arise, where servers in the cluster malfunction, such as due to hardware or software faults, data corruption or other reasons. In certain cases, these malfunctions may affect the distribution of the processing load in a negative manner, causing various latencies, errors, dropped requests and other issues. For example, one particular problem involving the least-connections algorithm may occur when a backend server reaches a fast-fail state, but still appears on the surface to be legitimately processing requests. In that scenario, the faulty server appears to have the least number of connections because it is responding and closing connections very quickly albeit in an erroneous manner. As a result, the faulty server is routed more and more requests by the load balancer, thereby further exacerbating the problem. These, as well as various other load distribution errors and server malfunctions, can significantly impede the performance of the server network and the various resources therein.